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Abstract 

For almost two decades the question of whether tabu search (TS) or simulated annealing (SA) performs 
better for the quadratic assignment problem has been unresolved. To answer this question satisfactorily, we 
compare performance at various values of targeted solution quality, running each heuristic at its optimal 
number of iterations for each target. We find that for a number of varied problem instances, SA performs 
better for higher quality targets while TS performs better for lower quality targets. 
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1. Introduction 

The quadratic assignment problem (QAP) is 
a combinatorial optimization problem first intro- 
duced by Koopmans and Beckman 12]. It is NP- 



hard and is considered to be one of the most diffi- 
cult problems to be solved optimally. The problem 
is defined in the following context: A set of N facil- 
ities are to be located at N locations. The distance 
between locations i and j is Dij and the quantity 
of materials which flow between facilities i and j is 
Fij. The problem is to assign to each location a 
single facility so as to minimize the cost 



N N 



= 1 j=l 



where p(i) represents the location to which facility 
i is assigned. 

There is an extensive literature which addresses 
the QAP and is reviewed in [H 0, [U [lj] . With 
the exception of specially constructed cases, opti- 
mal algorithms have solved only relatively small in- 
stances (N < 36). Various heuristic approaches 
have been developed and applied to problems typ- 
ically of size N w 100 or less. Two of the most 
successful heuristics to date for the QAP are tabu 
search (TS) and simulated annealing (SA). They 
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are basic heuristics which are used alone or as com- 
ponents in hybrid and iterative metaheuristics. 

Comparisons of the performance of SA and TS 
for the QAP have been inconclusive. In this work, 
we are able to successfully characterize the relative 
performance of these heuristics by performing the 
comparisons for various values of solution quality 
and by setting the number of iterations for each 
heuristic to the optimal one for the target solution 
quality. 

As is common practice, we define the quality, Q 
of a solution 

p. _ C — Cb es t 
^best 

where C is the value of the objective function for 
the solution and Chest is the best known value of 
the objective function for the instance. The lower 
the value of Q, the higher the quality. 

We find that for each problem instance, there is 
a value of Q, Q* , above which (lower quality) tabu 
search performs better — requires less time — than 
simulated annealing and below which (higher qual- 
ity) simulated annealing performs better. 



2. Background 

The tabu search heuristic for the quadratic as- 
signment problem consists of repeatedly swapping 
locations of two nodes. A single iteration of the 
heuristic consists of making the swap which most 
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decreases the total cost. Under certain conditions, 
if a move which lowers the cost is not available, 
a move which raises the cost is made. To ensure 
that cycles of the same moves are avoided, the same 
move is forbidden (taboo) until a specified later it- 
eration; we call this later iteration the eligible it- 
eration for a given move. This eligible iteration is 
traditionally stored in a tabu list or tabu table. The 
process is repeated for a specified number of itera- 
tions. 

The simulating annealing heuristic also consists 
of swapping locations of two facilities. In the simu- 
lated annealing approach used here [8j], each possi- 
ble swap is considered in turn and <5, the change in 
cost for the potential swap, is calculated. The swap 
is made if 8 is negative or if 

e- & ' T > r, 

where T is an analog of temperature in physical sys- 
tems that is slowly decreased according to a spec- 
ified cooling schedule after each iteration and r is 
a uniformly distributed random variable between 
and 1 . Randomly making moves which increase the 
cost is done to help escape from local minima. 

Pardalos [19f compared the performance of four 
algorithms including simulated annealing and tabu 
search and found that "all of these approaches have 
almost the same performance". Paulli 2ll] com- 
pared simulated annealing and tabu search and 
found that "when CPU time is taken into consid- 
eration, simulated annealing is clearly preferable to 
tabu search". On the other hand, [5] finds that 
"RTS (Reactive Tabu Search) needs less CPU time 
than SA to reach average results in the 1% [of the 
best known value] region". In 1998, summarizing 
the situation, Cela [7| commented that "There is 
no general agreement concerning the comparison of 
the performance of simulated annealing approaches 
with that of tabu search approaches for the QAP" . 
We are not aware of any later work which has clar- 
ified the issue. 

3. Approach 

We address the question of whether tabu search 
or simulated annealing performs better for the 
quadratic assignment problem by recognizing that 
the answer depends on desired solution quality and 
by: 

• defining a performance metric that ensures a 
fair comparison of different heuristics, 



• determining the optimal number of iterations 
for a given target quality for TS and SA for 
each problem instance; for a fair comparison 
of heuristics, it is critical to run each heuristic 
at its optimal number of iterations for a given 
target solution quality. 

• measuring the performance of TS and SA at 
multiple target qualities. 

4. Performance Metric 

To fairly compare heuristics, solution quality and 
time must be taken into account. Simulated anneal- 
ing and tabu search are multi-start heuristics; many 
runs of the heuristic are executed, each with a dif- 
ferent random starting configuration. A commonly 
used performance metric for multi-start heuristics 
is the percentage of these runs which attain a spec- 
ified value of the quality Q (typically 0.01). How- 
ever, this metric doesn't take run time into account. 
Sometimes, the runs times for individual runs of 
the heuristics are constrained to be equal but this 
is problematic because, as we show below, for a fair 
comparison each heuristic should be run at the op- 
timal number of iterations for the quality goal Q. 
One method of characterizing the performance of 
multistart heuristics with different run times em- 
ploys run-time distributions of the times needed 
across multiple runs to achieve a certain quality 
goal (see e.g. [2^, [![). Instead of using distribu- 
tions, we define the performance metric T(Q,I) as 
the average time to attain a quality goal of Q during 
a set of runs, each run with / iterations: 



f(Q,I) 



N(Q,I)' 



where t% is the CPU time for run i and N(Q, I) is 
the number of runs which attain a quality goal of 
Q or better. 

Because one heuristic may perform better de- 
pending on the quality goal, we calculate this per- 
formance metric not just for a single quality goal 
(e.g. 0.01) but for a range of quality goals. 

5. Numerical Results 

We use C++ implementations of SA and TS 
in the public domain to perform our computa- 
tional experiments. Both implementations are by 



Taillard, and are available at http://mistic.heig- vd 
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.ch/taillard/. The TS code implements the robust 
tabu search of [24|; the SA code implements the 
simulated annealing heuristic of Q. Both imple- 
mentations are straightforward and a few pages 
each in length. We run the TS heuristic with pa- 
rameter settings as described in 24] ) : tabu list size 
between 0.97V and 1.17V and aspiration function pa- 
rameter equal to 27V 2 ; there are no settable param- 
eters for the SA implementation. 

5.1. Determination of Optimal Number of Itera- 
tions 

Given a fixed time in which a heuristic can be 
executed, there is a tradeoff between the number 
of iterations per run and the number of runs which 
can be performed. The optimal number of itera- 
tions per run to reach a quality goal of Q, I op t(Q), 
is the value of I which minimizes T(Q,I). We de- 
termine I op t(Q)) as follows: For various values of /, 
Ii, we run each heuristic multiple times and calcu- 
late T(Q,Ii). Then, 



where 



I op t(Q) = {Ii\T(Q,Ii) =T(Q)} 



f(Q)=minf(Q,7i)- 



Thus T(Q) is the value of the performance metric 
when the heuristic is run at I op t(Q) iterations. 

In Fig. []Ja), using the TailOOa problem instance 
from QAPLIB [6J as an example, we illustrate the 
process of finding the optimal number of simulated 
annealing iterations for Q — 0.02,0.01, and 0.006. 
The optimal number of iterations, I op t, for each 
value of Q is the well defined minimum value of 
T for each plot. For a given value of Q, we note 
the large variation in T. We also note the large 
variation in I opt for the different values of Q. Thus, 
choosing a non-optimal value of iterations (e.g. a 
single value for the number of iterations for differ- 
ent Q) will result in an unfair characterization of 
the performance of the heuristic. Similarly, Fig. 
[ljb), illustrates the process of finding the optimal 
number of tabu iterations for the instance TailOOa 
for Q = 0.02, 0.015, 0.01 and 0.009. 

In Fig. Q](c), we pl ot Io P t, versus Q for SA and 
TS. For TS, I opt increases as Q decreases but does 
not increase below Q ~ 0.01. We infer that for TS 
there is no benefit to increasing the number of itera- 
tions below this point; any improvements in quality 
are gained by running more random starting con- 
figurations. On the other hand, SA benefits by in- 
creasing the number of iterations as Q is decreased 



over the complete range of Q studied. The subject 
of an optimal number of iterations for the quality 
goal Q = for simulated annealing is treated ana- 
lytically in Q. 

5.2. Performance Comparison of SA and TS 

We perform computational experiments on the 
following problem instances from QAPLIB [6] rep- 
resenting a range of problem difficulty, type and 
size. 

• TailOOa [24| is a totally unstructured instance 
consisting of random distance and flow matri- 
ces. 



25] 



are 



• nug30 [17(|, skolOOa |22j, and thol50 
instances in which the distances are the Man- 
hattan distances between locations on a grid. 

• lipa90a [14[ is a generated problem instance 
with a known optimal solution. 

• drellO [§] is a structured instance consisting of 
a "grid" flow matrix with non-zero entries for 
nearest neighbors only. It is part of a series of 
instances that are specifically designed to be 
difficult for heuristics. 

For each problem instance we execute a series 
of runs for various values of I including: I = 10 6 , 
5 • 10 6 , 10 • 10 6 , 50 • 10 6 , 100 • 10 6 , 500 • 10 6 , and 10 9 
for SA and I = 10 3 , 5 • 10 3 , 10 ■ 10 3 , 50 • 10 3 , 100 ■ 
10 3 , 500 • 10 3 , and 10 6 for TS. For some instances, 
additional values of I are used. As described above, 
for each problem instance we determine the value 
of T(Q) for various values of Q. 

By plotting T(Q) for each heuristic we can com- 
pare the performance of the heuristics when they 
are run with the optimal number of iterations. Fig. 
[5] plots T{Q) versus Q for the instances studied. 
Despite differences in details, they all share the 
characteristic that for each problem instance, there 
is a value of Q, Q* , above which (lower quality) 
tabu search performs better — requires less time — 
than simulated annealing and below which (higher 
quality) simulated annealing performs better. SA 
achieves lowest known costs for all but the TailOOa 
instance. TS achieves the lowest known cost for 
three of the six instances. 

In Table [1] we list the values of Q* for each of 
the instances studied. Note that if only the value 
Q = 0.01 were considered, the conclusion would 
be simply that SA is better for some instances and 
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TS for others. This explains why earlier studies of 
relative performance where not able to draw clear 
conclusions. 

5.3. Hardness of Problem Instances 

To compare the relative hardness of the problem 
instances studied, in Fig. [3] we plot T versus Q for 
the problem instances in a single panel. The rela- 
tive hardness of the instances for a given solution 
quality is given by the relative value of T at that 
quality. Comparing this figure with Table Q] note 
that Q* appears to be correlated with the hardness 
of the problem. With the exception of TailOOa, the 
harder the problem, the higher the value of Q* and 
thus the wider the range of Q in which SA performs 
better than TS. 

6. Discussion 

How do we explain our results that, for each prob- 
lem instance studied, there is a value of the quality 
Q, Q* , above which TS performs better than SA 
and below which SA performs better? A possible 
qualitative explanation is that TS essentially uses 
a steepest descent method to quickly find an ini- 
tial local minimum while SA finds the local min- 
ima in a more random way — sometimes mak- 
ing moves which increase the total cost even when 
moves which reduce the cost could be made first. 
Hence for high Q, TS performs better. Once a lo- 
cal minimum is found, however, SA is better able 
to escape and find a lower minimum. As opposed 
to TS, to attain better solution quality it is always 
better to run fewer SA runs with a higher number 
of iterations. 

Areas for future research might address the fol- 
lowing questions: 

• Is similar behavior observed when comparing 
SA and TS applied to other combinatorially 
complex problems? 

• When optimal numbers of iterations are used 
for SA and TS within such hybrid heuristics 
as hybrid genetic search, is the performance of 
the hybrid heuristic improved? 

• How does the performance of other heuristics 
(e.g. hybrid, iterated, ANT) compare when 
taking solution quality into account? 

• How are our findings changed if variants of TS 
are used? Can SA be modified to also outper- 
form TS at high values of Q1 
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Table 1: Value of Q, Q* below which SA performs better 
than TS. 



Problem 


Chest 


Q* 


nug30 


6124 [3] 


0.0014 


lipa90a 


360630 [14] 


0.0047 


skolOOa 


152002 [10] 


0.01 


tailOOa 


21052466 


0.0125 


thol50 


8133398 \1S\ 


0.025 


drellO 


2052 [9] 


0.058 
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Figure 1: (a)Dcpendence of T on number of simulated an- 
nealing iterations / for quality Q = 0.02,0.01 and 0.006 for 
the QAP instance TailOOa. (b)Dcpcndcnce of T on num- 
ber of tabu iterations / for quality Q = 0.02, .015, 0.01 and 
0.009 for the QAP instance TailOOa. (c) Optimal number of 
iterations, I op t versus Q for SA(squares) and TS (triangles). 
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Figure 2: T versus Q for various problem instances for SA(squarcs) and TS (triangles). For plots which achieve the lowest 
known cost for an instance (Q = 0), we extend the line connecting the plot points to the left edge of the panel. 
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Figure 3: T versus Q for for all problem instances studied: nug30 (squares), lipa90a (disks), skolOOa (up-pointing triangles), 
tho!50 (diamonds), tailOOa (right-pointing triangles, drellO (left-pointing triangles). 
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